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Abstract 

Robust  multivariate  prediction  and  interpolation  problems  for  statistically 
contaminated  vector  valued  second  order  stationary  processes  are  considered.  The 
statistical  contamination  is  modeled  by  requiring  that  the  spectra  of  the 
processes  lie  within  certain  nonparametric  classes.  Both  prediction  and  inter¬ 
polation  are  then  formalized  as  games  whose  saddle  point  solutions  are  sought. 
Finally,  such  solutions  are  found  and  analyzed,  for  two  specific  multivariate 
spectral  classes. 
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1. 


Introduction 


The  prediction  and  Interpolation  problems  for  stationary  processes  have 
received  considerable  attention  for  a  number  of  years.  The  bulk  of  the  work  con¬ 
centrates  around  scalar  processes  and  the  parametric  model.  The  assun^tion  there 
is  that  the  measure  generating  the  stochastic  process  Is  well  known.  The  Initial 
significant  results  on  prediction  and  interpolation  for  the  parametric  model  were 
given  by  Wiener  (1949)  and  Kolmogorov  (1941) . 

Strictly  speaking,  the  term  prediction  refers  to  the  extraction  of  a  datum 
from  the  process,  when  a  number  of  past  process  data  have  been  observed  noiselessly. 
The  term  Interpolation  refers  to  the  same  extraction,  when  past  as  well  as  future 
noiseless  process  data  are  available.  The  two  terms  are  extended  sometimes  to 
Include  noisy  observation  data.  Some  results  on  those  extended  problems,  and  for 
the  parametric  model,  can  be  found  in  the  papers  by  Snyders  (1973)  and  Viterbl 
(1965) .  We  point  out  here  that  the  majority  of  studies  on  the  extended  problems 
consider  asymptotic  and  linear  prediction  and  Interpolation  operations. 

The  la'st  few  years  considerable  attention  has  been  given  to  the  robust  extended 
prediction  problem.  Some  attention  has  also  been  given  to  the  robust  nonextended 
interpolation  problem.  The  robust  model  is  nonparametrlc ,  and  the  assunqition  is 
that  the  measure  that  generates  the  stationary  process  is  not  well  known.  The 
existing  work  on  robust  extended  prediction  and  interpolation  concentrates  atound 
scalar  stationary  processes,  linear  asymptotic  prediction  and  interpolation 
operations,  and  noisy  observation  data.  Representative  results  here  include  robust 
Wiener  and  Kalman  filtering  for  scalar  stationary  processes  (Masrellez  et  al  (1977), 
Kassam  et  al  (1977),  Martin  et  al  (1976),  Clminl  et  al  (1980),  Poor  (1980)).  Related 
work  on  time  series  outliers  can  be  found  in  Martin  et  al  (1977) .  Hosoya  (1978) 
considers  the  robust  nonextended  prediction  problem,  for  linear  contaminated 
scalar  stationary  processes.  The  robust  solution  Is  found  there  within  the  class 
of  asymptotic  linear  prediction  operations.  A  game  theoretic  formulation  on  the 


measures  of  the  stochastic  processes  is  presented  by  Papantonl-Kazakos  (1984),  for 
the  robust  extended  prediction  problem.  Chen  et  al  (1981,  1982)  consider  robust 
multidimensional  matched  filtering,  for  classes  with  Identical  eigenvectors. 
Regarding  the  robust  nonextended  Interpolation  problem,  for  scalar  processes,  the 
Interested  reader  may  look  Into  the  works  by  Tanlguchi  (1981)  and  Kassam  (1982) . 

The  prediction  problem  for  vector  processes  Is  considerably  more  involved  than 
that  for  scalar  processes.  The  difficulty  is  mainly  due  to  the  cross  correlations 
among  the  component  processes,  which  have  a  direct  Impact  on  the  complexity  of  the 
correlation  matrix,  and  the  spectral  distribution  matrix  of  the  vector  process. 
Important  questions  regarding  the  structure  of  a  vector  process  such  as  rank, 
regularity,  and  non-determlnacy  are  treated  by  Nlener  et  al  (1957),  (1958),  Helson 
et  al  (1958),  Hannan  (1970),  and  Zasuhln  (1941). 

In  the  present  paper,  we  consider  the  robust  nonextended  prediction  and  Inter¬ 
polation  problems  for  vector  stationary  processes.  Vector  processes  have  not  been 
treated  In  this  case  (some  limited  consideration  for  the  interpolation  problem  can 
be  found  In  Tanlguchi  (1981)) ,  and  they  present  interesting  peculiarities  both 
theoretical  and  practical.  We  will  generally  assume  that  the  spectrum  of  the 
vector  process,  which  Is  represented  by  the  spectral  distribution  matrix,  belongs 
to  a  class  of  spectra,  and  we  will  formulate  the  prediction  and  Interpolation  pro¬ 
blems  as  games  with  saddle  point  solutions.  Then,  we  will  find  those  solutions  for 
two  specific  spectral  classes.  One  of  the  classes  represents  linear  contamination 
of  a  nominal  spectral  matrix.  The  other  class  Includes  the  set  of  all  spectral 
matrices  with  fixed  energy  on  prespecified  frequency  quantiles. 

The  organization  of  the  paper  Is  as  follows.  In  section  2,  we  give  a  back¬ 
ground  on  the  multivariate  prediction  and  interpolation  problems.  In  section  3, 
we  define  the  spectral  classes  under  consideration,  and  we  formalize  the  prediction 
and  Interpolation  games.  In  sections  4  and  5,  we  find  the  saddle  point  solutions 


for  the  prediction  and  interpolation  games  respectively.  Finally,  in  section  6 
we  present  some  conclusions  and  a  brief  discussion. 

2.  Preliminaries 

The  original  linear  prediction  problem, for  an  n-variable,  discrete-time, 
second  order  stationary  process,  {^,  k  e  Z},  is  equivalent  to  searching  for  a 
matrix  trigonometric  polynomial  g^Ce^***)  of  the  form 

N 

gp(e^“)  -  I  +  e^“^  (1) 

i-1 

which  makes  the  mean  square  error  of  prediction 

tr  /  g  (e^“)d  F(a))  gf  (e^“)  (2) 

P  P  -v  ^  / 

as  small  as  possible.  In  (1),  N  runs  over  all  the  positive  Integers,  Aj^,A2  ...  are 
any  nxn  complex  matrices,  and  I  is  the  nxn  identity  matrix.  In  (2),  F(c»)  denotes 
the  spectral  distribution  matrix  of  the  process,  which  defines  a  positive  definite 
Hermltian  finite  matrix-valued  measure, on  the  Borel  field  of  the  measurable  space 
[-Tr,Trl,  with  J"  d  F(u)  ■  ••  E  fx^  x^  }.  The  symbols  *,  T,  and  tr  stand  for 

conjugate,  transpose,  and  trace,  respectively. 

Let  be  the  convex  set  of  polynomials  of  the  form  (1) .  We  define  the 
space  L2(dF(<»'))  of  all  nxn  matrix-valued  functions  A(u))  on  [-ir,Tr],  for  which  (3) 
below  is  true. 

tr  /  A(u)d  F(w)  A*  (w)  <  »  (3) 

-IT 

Then,  Spd  L2(d  F((*>)).  Considering  any  two  elements,  Aj^(u)  and  A2(“)»  of 

IT 

L2(d  F(io))  as  equivalent,  iff  tr  /  (Aj^(u)-A2(u))d  F(w)  (Aj^(u))-A2(w))  “  0, 

-IT 

then,  L2(d  F(oi)))  is  made  into  a  Hilbert  space  (Hannan  (1970)),  with  inner  product 
and  norm  defined  respectively,  as  follows. 


,■  V-V*  *'»  •  .  .  •-  *.  -  T.-  -- 


•  f  /;  /  ’  ^  '•  • 


V  ^ 

(A^(w),  “try  A^(u)  d  F(a))  A*  (u) 

—IT 


F(w)  “  A^(w)> 


1/2 


Under  the  new  notation,  (2)  can  be  rewritten  as  follows. 


i 


%<«“)■  i  P(«) 


(4) 


Now,  let  S  (d  F((u))  be  the  closure  of  S  In  L.(d  F(a))).  Since  S  (d  F(u))  is 
P  P  2  p 

a  closed  and  convex  sec  in  the  Hilbert  space  L2(d  F(b))),  it  contains  a  unique 
element  of  smallest  norm  (unique  in  the  equivalence  sense  defined  above) .  It 
follows  that  the  Inflmum  in  (2)  with  respect  to  is  attained  in  S^(d  F(u))) . 

For  reasons  that  will  be  explained  below,  we  are  going  to  consider  a  more 
general  prediction  problem,  by  enlarging  the  set  S^,  to  contain  all  matrix  tri¬ 
gonometric  polynomials  g^Ce^^  of  the  form. 


N 


1-1 


jU)i 


(5) 


1 


*■ - ^ 


A 


As  before,  N  runs  over  all  the  positive  Integers,  and  Aj^,  A2  ...  are  any  complex 

nxn  matrices.  A^,  however,  can  now  be  any  nxn  complex  matrix,  whose  determinant 

is  constrained  to  be  equal  to  1.  Let  be  the  set  of  all  polynomials  of  the  form 

(5).  The  convex  hull  of  S°  will  contain  all  polynomials  of  the  form 
N  P  P 

B  +  Z  B.  e^***^,  with  det  (B  )  >1.  This  follows  from  the  concavity  of  the  func- 
O  1  o  —  ' 

tlon  det  (•)»  namely,  det  (XA+(1-X)B)  ^  (det(A))^(det(B))^”^;  0  £  X  £  1  (Bellman 

(1970)).  If  we  take  the  closure  S°*^(d  F(u>)  of  in  L»(d  F(u))),  we  can  define 

P  p  / 


the  new  prediction  problem  as  follows, 
min 


gJ(eJ“)eS°‘'(d  F(u))) 


ep(F(a,),g°(e^“))  -  (2ti)‘^ 


min 


g®(e^“)£S°‘'(d  F(u))) 


rto) 


(6) 


4 

i 


.V 


As  a  result  of  the  derivation  of  the  optimal  predictor  in  Relson  et  al  (1958), 
the  minimum  in  (6)  assumes  a  closed  form  expression.  In  particular, 

TT 

min  e  (F(w) ,g°(e^*^))  =  n  exp[(27rn)  ^  f  tr<togf (a))du)]  (7) 

P  P 

g^(e^“)eS°‘'(d  F(co)) 

where  f(b))db)  is  the  absolutely  continuous  part  of  the  measure  d  F((>)),  with  respect 
to  the  Lebesque  measure  in  [-ir,^],  and  where  f(u)}  is  the  spectral  density  matrix 
of  the  process,  which  is  Hermltlan,  nonnegative  definite  a.e.  (do)),  and  integrable 
in  [-'ir,'ir].  If  the  scalar  function  tr-Cogf(ii))  in  (7)  is  not  integrable  (since 
IT 

S  tr£ogf(u)da)  is  bounded  from  above, as  can  be  verified  by  using  Jensen's  inequality 

this  can  only  happen  if  y  tr^og  f(a))da»  =  -»),  the  right  hand  side  of  (7)  is  inter- 

-ir  ,  - 

preted  as  zero.  An  element  g^  ((o)  in  (dF(u)),  that  attains  the  minimum  in  (6) 
is  such  that, 

g  (w)(g°  (w))  =  exp[(2‘irn)  f  tr£ogf(u))dw  I-^gf(w)J  (8) 

^  -TT 

for  points  u)et-iT,ir] ,such  that  f(a))  exists  (i.e.  a.e.  (do))),  and  g®  (w)  =  0  for  an 

P 

at  most  countable  subset  of  [-11,11].  The  latter  is  simply  a  manifestation  of  the 

fact  that  the  singular  part  of  d  F(u)  contributes  nothing  to  the  minimum  in  (7), 

and  it  corresponds  to  a  purely  deterministic  process.  If  g°  (u>)  exists,  it  is 

then  proven  by  Helson  et  al  (1958)  that  g'^  (a))e  S°  (d  F(u)));  i.e.  the  determinant 

P  P 

of  its  leading  Fourier  coefficient  is  equal  to  1,  or  equivalently  the  lll^n^1^nlm  in 

(6)  for  g°(a))e  S°  (d  F(a)))  exists,  although  S°  (d  F(a)))  is  not  convex,  and  it  is 
P  ^  P  P 

attained  at  g°  (b>) . 

oc 

Consideration  of  the  prediction  problem  in  (d  F(u))),  or  equivalently  In 
(d  F(ca)),  has  the  remarkable  advantage  of  a  simple  closed  form  expression  for 
the  minimum  error  given  by  (7),  which  is  a  direct  generalization  of  Szego's,  formula 
(Grenander  et  al  (1958)),  for  scalar  processes. 


The  linear  interpolation  problem  for  an  n-varlable  discrete-time  second  order 
stationary  process  {3^,  k  e  Z}  is  less  difficult,  because  the  constraint  set  of  the 
associated  minimization  problem  is  larger  than  (d  F((«))),  since  no  causality 
requirement  exists.  We  will  denote  by  the  convex  set  of  all  trigonometric  poly¬ 
nomials  of  the  form, 

N 

g^(eJ“)  =  1+ 

1— N 
1^0 

where  N  runs  over  all  positive  Integers,  and  {A^,  i  0}  run  over  all  complex  nxn 
matrices.  The  error  that  has  to  be  minimized  is  similar  to  that  in  (2),  and  it 
is  rewritten  here  for  completeness. 


e^(F((u)  ,g^(e^“))-(2Tr)"^  tr  /  g^(e^“)d  F(a))g*’'(e^“)=(2ir)"^|  |g^(e^“)  |  (10) 

Taking  the  closure  (d  F(<ii))  of  in  L^Cd  F(u>)),  we  can  define  the  interpolation 
problem  as  follows. 


min_  e.(F(<»)),g.(e^“))=(2Tr)"^  min_ 

g^(u)eS^(d  F(u)))  gj^(w)  S^(d  F(iD)) 


|gi(e 


jO), 


))ir 


dF(u)) 


(11> 


As  derived  in  Hannan  (1970),  the  minimum  in  (11)  is  given  by 

Ti  _  _1 

min  e^(F(u))  ,gj^(w))  =  2tt  tr  I(  y*  f  (u)du))  J  (12) 

g^((o)eSj^(d  F((*)))  ^ 

and  it  is  attained  for  some  g^(u)e  (d  F(a))),  such  that 

-1  -I 

g'(aj)  -  2ir  (yf  ■‘^(a))da))  f  ^(u)  ;  a.e.  (dw)  (13) 

-IT 

g^(u))  »  0  for  an  at  most  countable  subset  of  I -it,  ti],  where  the  singularities  of  the 
spectrum  are  located .  In  (13) ,  f”'*’  (u)  is  the  Penrose-Moore  generalized  inverse  of 


f((«))  and  it  is  integrable  for  full  rank  processes. 


7 


3.  The  Robust  Formalization 

We  now  look  at  the  above  problems  from  a  different  point  of  view.  We  assume 
that  the  spectral  structure  of  the  process  Is  only  vaguely  or  Incompletely  specified. 
This  corresponds  to  a  more  realistic  situation,  since  the  procedures  for  obtaining 
the  spectrum  of  a  process  always  Involve  errors.  This  applies  even  more  to  vector 
processes,  where  the  Increased  complexity  results  in  larger  errors.  With  the  above 
in  mind,  it  is  clear  that  new  formalizations  of  the  problons  considered  in  section 
2  are  needed.  Such  a  formalization  is  given  below,  where  the  spectral  distribution 
matrix  of  the  process  is  assumed  to  be  a  member  of  a  whole  class  of  spectral  distri¬ 
bution  matrices.  For  the  purpose  of  this  work  we  are  going  to  consider  two 

different  types  of  spectral  classes,  denoted  by  F  and  F-,  which  are  defined  as 

L 

follows: 

(a)  F,  =  {F((o)/F(a))  =  (l-e)F  (to)  +  e  H((o),  toe  [-it, it], e  fixed  and  such  that, 

L  o 

0  <  e  <  1. 

F  (to):  well  defined  fixed  nominal  spectral  distribution  matrix 
»  o 

H((o) :  arbitrary  spectral  distribution  matrix  satisfying 

—1  ^  —1  ^ 

(2Tr)  ^  tr  /  d  H((o)  =  (2Tr)  tr  /  d  F  (to)  =  W  >  0.  W  fixed} 

-IT  -IT  ° 

(b)  F  =  {F(to)/(2Tr)''^tr/  d  F((o)  =  c  >  0,  i=l,...k,  c  ,...c  fixed 

4  1  IK 

D^,...D^  fixed  Lebesque  measurable  subsets  of  [-ir,ir]  with  positive 
measure  each  and 

k 

D  n  D  =  0;V  1  ^  j,  U  D  -  I-ir,Trl} 

^  J  i=l  ^ 

F^  is  called  the  e-contamlnated  class,  or  the  gross  error  model,  and  it  corresponds 
to  the  situation  where  the  nominal  process  occurs  only  with  probability 

1-e  <  1,  while  with  probability  e,  any  other  process,  with  the  specified  energy 
constraints,  may  occur.  F^  is  called  the  p-polnt  class,  and  it  contains  all  the 


spectra,  whose  energy  Is  specified  by  a  positive  number,  on  a  finite  collection  of 
mutually  exclusive  and  exhaustive  measurable  subsets  of  [-Tr,Tr],  with  positive 
Lebesque  measure. 


In  pursuing  a  robust  formalization,  for  prediction  and  interpolation,  it  will 
be  necessary  to  restrict  the  classes  (d  F((d)),  (d  F((o))  defined  in  section  2 
for  the  simple  reason  that  Instead  of  a  single  F(b)) ,  a  whole  class  of  spectral 
matrices  is  considered.  In  particular,  we  will  consider  the  following  classes  of 
predictors  and  Interpolators. 


S°  (d  F(w)) 
F(a))$Fj^  P 


r\  S.  (d  F(u»;) 
F(a))eFj^  ^ 


O  S°  (d  F(w)) 
F(«*))eFQ  P 


n  S,  (d  F(a))) 

F(a))eFjj 


We  are  now  in  a  position  to  formalize  the  following  games: 
Find  pairs  (F®(u)  ,g®^(w))eF^xSj^j,(T  *=  L,Q.  R  p,i)  such  that 

» 

ej^(FT(u),8*T(“))  1  ej^(F®(a)),g^j(aj))  <  ej^(F®(u),gj^^(a))) 


(14) 


;  V  F^(u))eF^,  V  gRy(«)e 


(15) 


If  the  pairs  with  the  superscript  e  exist,  we  call  them  robust,  and  they  are  the 
saddle  point  solutions  of  the  games. 

In  section  2,  we  saw  that  the  minimum  error  expressions  (7)  and  (12)  depend 
only  on  the  absolutely  continuous  part  of  the  spectral  distribution  matrix  F(fa)) ; 
namely,  the  spectral  density  f((i)).  The  spectral  singularities  contribute  nothing 
to  the  minimum.  However,  under  the  present  formalization  it  is  not,  in  general, 
true  that  the  operators  gpj(<^)  will  belong  to  the  corresponding  Intersection 
classes  (T  ■  L,  Q,  R  ■  p,  1),  if  we  allow  spectra  with  singularities,  in  our 
classes  F.  ,  F-.  As  we  will  see  in  sections  4  and  5,  this  is  due  to  the  fact  that 

li 

the  operators  gpm(u)  are  defined  a.e.  (dut)  in  [-ir.Tr],  and  they  do  not  necessarily 
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have  to  be  limit  points  of  the  corresponding  sets  of  trigonometric  polynomials, 
with  respect  to  norms  L,(d  F(a))),  for  all  the  F(a))'s  in  either  F  or  F  ,  that 
contain  singularities.  For  this  reason,  and  throughout  the  rest  of  the  paper,  we 
will  restrict  attention  to  processes  with  absolutely  continuous  spectra.  In 
particular,  we  will  assume  that  the  nominal  spectral  distribution  matrix  F^(ai),  in 
the  definition  of  the  class  is  absolutely  continuous  with  density  f^Cw) ,  and 
that  H(aj)  runs  over  the  absolutely  continuous  spectra  with  density  denoted  by  h(a)) . 
Similarly,  the  class  F^  will  consist  of  all  F(aj)  that,  in  addition  to  the  constraints 
imposed  in  definition  (b) ,  will  be  absolutely  continuous  as  well.  Under  those 
assumptions,  the  two  classes  F^  and  F^  reduce  to  classes  of  spectral  density  matrices. 
For  notational  simplicity,  we  will  frequently  omit  arguments  of  functions.  In  sec¬ 
tion  4,  below,  we  solve  the  prediction  games  on  F_xS  ,  and  F.xS  In  section  5, 

L  pL  Q  PQ 

we  solve  the  interpolation  games  on  F  xS.^  and  F  xS  .  In  all  cases,  we  state  the 

L  Q  xQ 

solutions,  and  we  prove  them  directly  by  construction, 

4.  The  Solution  of  the  Prediction  Games  on  F.xS  .  and  F..XS  .. 

- - ^L— pL - (^-pQ 

Let  {X°(u),  2^(0));  i=l,...n}  be  the  ordered  eigenvalues  of  (X°(u)^X°^j^(a)) , 

i=l, . .  .n-l,V(i»e[-Tr,Tr]) ,  and  the  corresponding  normalized  eigenvectors.  Since 
—1  ^  —1  ^  n 

(2tt)  tr  J"  (l-e)f  (b))d(i)  =  (Ztr)  y  Z  (l-e)X.  (u))da)  =  (i-e)W<W,  there  exists  a 
-ir  °  -IT  1=1 

positive  number  c  such  that. 


1  ' 

(2it)  f  max((l-e)X?((A)),  c)du)  =  W 


Let  us  define  the  set  of  functions. 


0  o 

Xj^(u))  =  max((l-e)X^(aj) ,  c),  i=l,...n 


and  the  matrix 


(18) 


f  (<t))  is  HermiClan  and  positive  definite,  for  all  (()c[-Tr,7r} ,  since  its  smallest  eigen- 
L 

value  is  uniformly  larger  than  c  >  0.  Furthermore,  f.  (a))eF^,  since  f_  (a>)-(l-e)f  (a))  = 

L  L  L  O 

^  C  O  O  O 

“  E  (X.(u))  -  (l-e)X.((o))  Jc? (u>)  (oj) )  is  nonnegative  definite  for  all  O),  and  (16) 
i»l  1-1-1 

holds.  We  also  note  that  -°»<n^og  c  =  tr  >tog(c.l)  £  tr  £og  ff(a))  and 
_1  IT  u 

(2Trn)  J'  tr  tog  ff  (u))d(a  £  ^g((2im)  S  tr  ff((»))dw)  =  tog  —  <  »,  from  which  we 

-IT  -IT  . 

A  6  6  —X 

conclude  that  tr  tog  f  (u))  is  Integrable.  Therefore,  tr  tog  (f  (u))  is  also  integrable. 

ij  L 

Finally,  since  0  <  (ff(a)))  ^  £  c  ^  I,  (ff(o)))  ^  is  also  integrable.  Let 

=  exp[(2irn)  y  tr  £og  fj^((»))d(»i] .  We  consider  the  matrix  K^j^(f®(uj))  ■*■,  which 

—It 

is  easily  recognized  to  be  equal  to  the  right-hand  side  of  equation  (8),  in  section 

2,  for  fj^  =  f,  and  which  satisfies  the  requirements  of  Theorem  7.13  In  Wiener  et  al 

(1957,  1958).  From  that  we  conclude  that  there  exists  a  factorization  of 
e  -1 

(f^  (it)))  of  the  following  form. 

gpL(‘^)(8pL(‘^))*’^  =  K^j^(f®(0)))“^  ;  a.e.(dw)  (19) 

where,  if  {A®,  n  e  Z}  are  the  Fourier  coefficients  of  g®  (oi),  then  A®  =  0  for 
n  pL  n 

n  <  0,  and  d*et  A®  =  1.  According  to  the  derivations  in  Helson  et  al  (1958),  g®  (oj) 

o  pij 

is  the  element  of  S®  (ff((*))du))  that  minimizes  e  (ff(ta)),  g  .(u)),  with  respect  to 
_  P  L  p  L  pL 

g  ,  (u)  e  S°  (ff  ((i))du))  .  That  is, 
pL  p  L 


SpL(“))  <^p(f^w),  gpj^(a))) 

;  V  g  (u))  e  S°  (ff  (a))da)) . 

pL  p  L 


In  Lemma  1  below,  we  prove  that  Sp^C*^)  ^  ^pL'  X  establishes  that  the  pair 

(ffiwi),  g%(a)))  is  the  solution  of  the  prediction  game  on  F  xS  . 

JL  PL  *j  pXa 

Lemma  1  (u)  c  S  . 

pL  pLi 

Proof 

e  -1  -1  e 

From  (19)  and  (f  (u))  <  c  I,  we  conclude  that  each  entry  of  g  (u)  is  essentially 

1*  pL 

G  0 

bounded  (du).  From  its  Fourier  coefficients  A^,  Aj^,...,  we  form  the  sequence 
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{G®(e^***)}  of  the  Fejer-Cesaro  partial  sums. 


where 


S®(e^“)  =  ^  A®  e^*^^  .  Evidently,  G®(e^“)  e  S°. 


By  the  usual  theory  of  this  sum,  each  entry  of  G^Ce^'^)  converges  a.e.  (do)  to  the 

N 

corresponding  entry  of  boundedly,  since  gpj^^^**)  is  bounded.  Put 

hjj(a))  =  G®(e^^)  -  gpL^“^*  Then,  for  any  f(a))eFj^  we  have: 

ll^j<“>llf(a,)da>"  ^  trhjj(u>)  f(a))  hj'^(a))da)  < 

—It 


<  f  X^^^(hj^^(tj)h,j(h)))  tr  f(u))d(»). 
-n 


where  denotes  maximum  eigenvalue.  Since  b^((>>)  0  a.e.(du))  boundedly,  it 

is  implied  that  (h^*^ (cj)  hjj(u»))  -*■  0  a.e.  (du)  boundedly.  Now,  since  tr  f(u)) 

contains  no  singularities,  due  to  the  assumed  absolute  continuity  of  the  menbers  of 
Fj^,  it  is  concluded  that  X^^^(h^'^(oj)  hj^Co)))  ->  0  a.e.  (tr  f(u))du>).  Application  of 
the  dominated  convergence  theorem  on  X^^^(h„  (h))  hjj(t»)))  yields: 

^  *T 

•tLm  f  X  (hJJ^(u)  h  (u))  tr  f(u)d(i)  = 

_ _  _  lIl3X  W  W 

N-+oi>  -TI 


f  -Clm  X  (h„  (u)  Ir  (u))  tr  f(a))  do)  =  0,  which  implies 
-Tf  H-x»  ^  " 


1 I  If(aj)dta)  ° 


The  preceding  arguments  show  that  there  always  exists  a  sequence  of  elements 
of  Sp,  which  tends  to  gp^^***) »  under  any  norm  1 1  *  I  If  Thus 
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(u)  e  S 


pL 


Remark.  The  basic  constituents  for  the  proof  of  lemma  1  are:  1)  The  fact  that  the 

eigenvalues  of  f ,  (u)  are  bounded  away  from  zero,  which  Implies  the  a.e.  (do))  bounded- 
L 

ness  of  g  -  ((t>) .  2)  The  absolute  continuity  of  the  members  of  the  class  F, ,  which 

pL  L 

permits  the  transition  from  the  a.e.  (du)  to  the  a.e.  (tr  f((t))  dot)  convergence.  The 
above  requirements  are  satisfied  for  all  the  other  games  we  consider  in  the  sequel. 

Theorem  1 

The  pair  (f ,  (u) ,  g  _  ((u) )  is  a  saddle  point  solution  of  the  prediction  game  on 

1j  pli 

F_  xS  _  . 

L  pL 

Proof 


He  have  to  prove: 


«L-  *pL>  ^  i  ‘p«L-  V  '  ’'l  "  ‘  »  'pL  " 


.e  e 

p'\’ 


p'  L’ 


*pL  pL 


(22) 


The  right-hand  side  inequality  in  (22)  follows  from  (20)  and  (f^  d(i))3S^^.  Also 


ep(f®,  gpj^)  «  n  expI(2Trn)  ^  f  tx  log  f^(u)du)]  =  n 


.-1 


-IT 


(23) 


and 


“  (2Tr)“^  f  tr[g®j^(g®j^)*  fj^Jdw 

-u 


p'  L’  "pL 
Combining  (24)  with  (19),  we  get. 


(24) 


«pL>  ■ 


(2Tr)"^  KeL  J  (x°(u)))**  fj^(u))  xj(a))da) 

1-1 


.-1  Oy. 


*T 


(25) 


Put  M.  (w)  -  (Xj(u)))  f ,  (w)  j^(u).  Since  f _  (u)  C  F,  ,  f _  (u)  -  (1-e)  f  (u)  should  be 

i  — 1  L  L  L  L  o 


nonnegative  definite  which  implies  that 


(26) 
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y^(oi)  5^  (1— e)  Xj^(w)  i  V 


Also 


(2#)"^  f  tr  fj^(w)du  =  (2Tr)“^  f  ^ 


-1 


(27) 


-TT 


1=1 


From  (25)  and  (23)  we  obtain. 


Sl>  -  %<•  *pt>  "  ■‘e.  4  ?  ^ 


1-1 


n  r 


2ir 

1-1 


y^(w)-(l-e)X°(a)) 
(l-e)X^((o)^c  (l-e)X°(a») 


y*  - i - dw  + 

/I  ^\\0/,.A  (l-e)X"(u))<c 


/ 

>o. 


y^(a))-c 


db> 


Kt  " 


—  2n  ^ 


1=1 


f 


y.(<»>)-(l-e)^4(“) 


du  + 


/ 


y^(u))-c 


d(D 


(1-e)  X°(aj)  >c 


(l-e)Xj^(uj)<c 


K  "  TT  y^  (w)-Xf(u)  K  If  g 

2f=E  /  *■  C-  —  ^  ^  tr(f^(«.>-f^(«.))d«.  -  0 

I-l  ■” 


and  the  left-hand  side  Inequality  in  (22)  follows. 


We  now  proceed  to  the  solution  of  the  prediction  game  on  F^xS^^.  We  define 
the  spectral  density  matrix 


,*?«“> E'l 


(28) 


i=l 


where  1  (u)  is  the  indicator  function  of  the  set  D.,  and  m(*)  is  the  Lebesque 

®i  g 

measure  in  [-ir.Tfl.  It  can  be  seen  by  inspection  that  fQ(«*>)  e  F^.  Since  c^^  >  0, 


m(D^)  >  0,  the  eigenvalue  of  fQ(‘^)  bounded  away  from  zero  and  from  infinity,  by 


.  .  .*• ■  v -.'av.v  - 


“  min  )^*  tT  respectively.  It  follows  that  (f®(u)))" 

i“l»»».lt  i  i*lj  • .  .k  i  '* 

and  tr  Zog  (£®(ii>))  both  exist  and  are  integrable.  We  thus  conclude  that  there 
exists  some  gpQ(w)  e  S®  (f®(ai)du)),  such  that 

8pQ(8pQ>*  -  (29) 

-1  ^ 

where  k  -  exp[(2irn)“^  r  tr  £og  f®  du] . 

eQ  Q 

where  the  Fourier  coefficients  of  g^^,  lA^,  1  e  Z},  vanish  for  i  <  0,  and  where 
0 

det  1.  Exploiting  the  assumption  of  the  absolute  continuity  of  all  the 

members  of  F^,  we  can  argue  exactly,  as  in  Lemma  1,  and  establish  that 

/  i  do)).  Also,  g®  is  the  element  of  S°  (f®  dw)  which  minimizes 

PW  '  P  PQ  P  Q 

e_(f^,  g  n).  We  conclude  this  section  with  the  following  theorem. 

P  Q  PQ 

Theorem  2 

The  pair  (f®,  g®p)  given  by  (28),  (29)  is  a  saddle  point  solution  of  the 
prediction  game  on  F^xS^^. 

Proof.  We  have  to  prove  that,  g®^)  <  ep(f®,  g®^)  <  ep(f®,  g^^^) 

;  V  fg  e  F^;  V  g^^  e  The  right-hand  side  Inequality  follows  from  the  fact 

that  S  _ClS°  (f®  du),  and  that  g®„  is  the  minimizing  element  of  e  (f®,  g  „)  for 
PQ  P  Q  pQ  P  Q  pQ 

g  ^  e  S°  (f®  du).  We  thus  have: 

PQ  P  Q 

•p«Q-  Sq*  ■  S  “  fq<*pq>*^  ^  - 


k  TT  k  _  nm(D.) 

^  f  tr[f„(f®)”^l  du  -  -5^^  y]  Tn  ..  /  tr  f„  du 


The  Solution  of  the  Interpolation  Games  on  F, xS..  and  F_xS 


We  start  with  a  proposition: 


Proposition  1 


If  p(u))  is  any  nonnegative  integrabie  function  in  [-ir.ir],  then  the  function 


T(Y)  =  Y  y*  tniax(Y,  p(w))l  do) 

-IT 


defined  for  0  <  Y  ^  ®ss  sup  p((o)  is  monotonic  and  continuous. 


Proof 


For  0  <  Y2  ^  Yj^  5.  pC^*!)  we  have: 

(I) 


T(y,)-T(y,)  =  y*  do)  +  /  do)  -  f  du)  -  /  du) 


Yj^23>(w)  Yj^<p(w) 


S  dart-  y* 


Y2^(li»)  Y2<p(‘*») 


do)  -  y 


Yjl^(w)>Y2  Yi<p(w)  Yj^<P(w)  **^‘*^^  Yj^(w)>Y2 

J  ^r.'S  d<«>  +  *  _  ,T\ —  du  >  0. 


Yj^<p(u) 


Yi^(u)>Y2 


^1^^2  /• 

Also,  T(Y|)-T(Yo)  £  — - —  •'  do)  ,  from  which  the  continuity  follows. 

^2  p(a))>Y2 


Now,  as  in  section  4,  let  {X^(u),  x®(a)),  i“l,...n}  be  the  ordered  eigenvalues 
and  the  corresponding  eigenvectors  of  the  nominal  spectral  density  matrix  f^^Cu). 
For  each  eigenvalue  we  define  the  function, 

T.(y)  ”  Y  y  [inax(Y,  (l'-e)X®(a)))]“^  du),  i“l,...n  (29) 

-IT  * 


Due  to  the  monotonlclty  and  continuity  of  T^(y)*  for  any  positive  number 
c  <  2it,  there  exists  a  unique  Yj  such  that  T^CYj)  “  c»  i“l,...n.  We  put 
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Y^(c)  =  t”'*’(c).  The  Inverse  mapping  T^^(c)  is  also  monotonlc  and  continuous.  Now, 


since  (2it)'"  y  E  (l-e)X?(uj)du  =  (l-e)  W  <  W,  there  exists  a  positive  number 
-IT  1=1  ^ 


c  .  such  that. 


(27t)"^  f  maxd^^Cc*),  (l-e)xj(a)))  du  =  W 


Before  we  proceed  further,  we  will  make  an  assumption,  concerning  the  eigen¬ 
vectors  of  f  ((!)),  {x^(a>),  l>l,...n}.  For  the  purpose  of  obtaining  closed  form 
o  1 

solutions,  we  will  assume  that  are  constant,  independent  of  u,  for  every 

1=1,... n.  We  denote  them  by  ^  omitting  their  argument.  Thus,  we  consider  the 
class  of  spectral  density  matrices, such  that  the  nominal  has  constant  eigen¬ 

vectors.  We  note  that  this  is  different  from  requiring  that  all  members  of  F^  have 
constant  eigenvectors. 


We  define: 


X®(a))  =  max(T‘^(c*),  (l-e)Xj(a))) 
n 


The  eigenvalues  of  f^ (u)  In  (32)  are  bounded  away  from  zero,  since  they  are  all 

la 

uniformly  larger  than  or  equal  to  T^^  (c  )  >  0.  Also,  e  Fj^,  due  to  (32),  and 

to  the  fact  that  f®(a))  >  (l-e)  f  ((a);  V  wet-ir,!:].  We  define 

—  o 


g®j^(a»)  -  2Tr|y  (f®(a)))"^du)  ^  (f®(u)))"^ 


which  minimizes  1 1 (**>)  1 1 ^e^j^ t  g£(u)  e  (f®du).  Since  (21:)  ^  f  g®j^(a))dw  ■  I,  the 

L  e 

Fejer-Cesaro  partial  sums  of  the  Fourier  series  of  trigonometric  poly- 

nomials  belonging  to  S^.  Furthermore,  since  the  entries  of  bounded  by 

(33),  the  sequence  of  the  Fejer-Cesaro  sums  will  converge  domlnatedly  a.e.  (doi)  to 


-V..- 


•-.r.  •• 


gi^Cu) .  The  absolute  continuity  of  the  members  of  the  class  together  vith  the 
application  of  the  dominated  convergence  theorem  then  Implies,  in  a  way  similar  to 
that  in  Lemma  1,  that  the  above  sequence  will  converge  to  g?. (u).  In  the  norm 

1 1*  I  lf(y)d(*,«  ^  ^L*  ^  ®iL  “  ^  ^  (f(i»>)du).  We  now 

L 

state  the  solution  of  the  game  on 


Theorem  3 

The  pair  (f^(a»),  g^^  (cj))  defined  by  (31),  (32),  (33)  Is  a  saddle  point  solution 
of  the  Interpolation  game  on  F^xS.^. 

Ij  XIj 

Proof 

We  restate  the  theorem,  as  follows. 


1  e^(f®((o),  gjL(w))  <.  ej^(f®(t»)),  gjj^(w)) 


;  V  fj^(a))  e  Fj^,  V  g^(w)  e 


The  second  Inequality  follows  immediately  from  section  2,  and  the  fact  that 
I  ^  e  ”1 

Put  k'  “  \  y*  (f. (u))  duj  .  The  following  relationships  are  valid: 


eL  '  '‘L 

-tr 


ei(fj^(a)),  g^ («»>))  -  e^(f®(a)),  g®j^(w))  »  (2Tr)  ^  y  tr  g®j^(u)) (fj^(u))-f®(u)) (g^(w))*  dw 


"  -1 

2tt  f  tr[k'j^(f®((»)))  (f^  ((D)-f®(a)))]du 


L'""  ‘L' 


- - - 

^  e  -1  2 

1=1  [y  (A®(u))  ^du)] 

-IT 


TT  xj  f  (U)  ^-X®(U)) 

/  ^  -‘■e  ^ 

-ir  (X®(w))^ 


T  T 

Since  fj^(u))  £  2l  (l^e)  ^g^w)  h.  (1“£)  ^^o^***^  —1 

■  (l-e)X. (u),  and  (2ti)  ^  f  tv  f  (b))d(d  -  (2Tr)”^  /  2  f(w)  x.  du  ■  W. 

^  -V  ::7r  1-1 


g.,  ((i>).  The  absolute  continuity  of  the  members  of  the  class  f  ,  together  with  the 
IL 

application  of  the  dominated  convergence  theorem  then  Implies,  in  a  way  similar  to 
that  in  Lemma  1,  that  the  above  sequence  will  converge  to  g?.  (<«>),  in  the  norm 

Xu 

Il*||f(^)dy.  for  any  f(u))  e  Thus  ^  ®iL  “  ^  We  now 

f  (u)  £  Fj^ 

state  the  solution  of  the  game  on  « 

Theorem  3 

The  pair  (ff(a)),  g?,  (w))  defined  by  (31),  (32),  (33)  is  a  saddle  point  solution 

li  Xli 

of  the  interpolation  game  on  F^  xS^^  . 

Proof 

We  restate  the  theorem,  as  follows. 

le^(f®(aj),  g^(«))  <.e^(f®(w),  g^(a))) 

;  V  fj^((o)  e  Fj^,  V  gjj^(a))  e 

The  second  inequality  follows  immediately  from  section  2,  and  the  fact  that 

Put  ■  ( y*  (f*(u))  •  The  following  relationships  are  valid: 

—IF 

e^(fL(«),  g^(w))  -  e^(f®(u),  g®L(<-))  *  (2^)"^  J  tr  g®j^(a)) (fj^(a))-f®(a))) (g^(aj))*’^da) 

-TT 

IT 

-  2Tr  y  tr[k^j^(f®(a)))  (fj^(a))-f®(w))lda) - 

-V 


£ 


,  e  -1  2 

i«=l  ly  (X®(aj))  ^du] 

“TT 


u  xj  f-(u))  x^-x®(u)) 

/  \  ^ - dU) 


-IT 


(X®(U)))^ 


(34) 


Since  f,  (u)  e  F,  ■>  f_  (u)  >  (l^e)  f  (u)  ->  xj  f,  (u)  ^  >  (1-e)  ^  f  (u)  x.  - 

L  L  li  O  “1  L  — 1  —  ”1  O  — 1 


IT 


-  (l“e)X?(<«)) ,  and  (2tt)  ^  y  tr  f.  (u)d<i)  -  (2“^)  ^  f  S  xt  f(w)  jc.  dw  ■  W. 

^  -  **  -IT  i-1 


-IT 


Put  V .  =  ^  £,(<»»)  X.  >  (l-e)X.(u));  V  we[-Tr,TiJ,  Then,  from  (34) 

1  ”1  L  — —  1 


we  get: 


g^L^w))  -  e^(f®(u)),  g®j^(w)) 


2„V _ i _  _  /  Vito)-Tt^c*) 

1-1[/  (X*<(u))"*auJ^  )>(l-e)X°{u)  (Tj^(c*))^ 

-7T 

v,(a))-(l-e)A®(a)) 

I  a  - o - 2 -  ^  ^ 

V(c*)<<l-e)X°(w)  ((i-e)Xj(u>))'^ 

<2. i:  [  y  , 

i=l  |^T^^(c*)>(l-e)X°(w)  (T^^(c*))^ 

v.(a))-(l-e)X®(u) 

1  «  -  -  T  ^  -  2  -  dw  - 

V(c*)<(l-e)Xj(u)  (T^^(c*)) 


"  ^ o  /  <v.(w)-x?(u))  dw  -  0. 


The  proof  is  now  complete. 


Finally,  we  examine  the  Interpolation  game  on  result  here  Is 

summarized  In  a  theorem. 

Theorem  4 

6  G 

The  pair  (fqCu),  g^^qCu))  which  is  defined  by  the  expressions, 
k 

fq(w)  ■  ^  27r  c^  Ijj  (u)  m”^(D^)*I  -  X*(w)*I 

£-1  ^ 


“  2ti 


(}ut 


(w))"^  dui’^  (f*(w))"^ 
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is  a  saddle  point  solution  of  the  interpolation  game  on 
Proof 


0 

Since  fqCu)  is  bounded  away  from  zero  and  from  infinity,  due  to  >  0, 
mCD^)  >  0,  i.  >  l,...k,  and  since  we  are  considering  absolutely  continuous  spectra 
only,  it  will  follow  again  that  S^qt***)  £  S^q.  The  inequality, 

ei(fq(a)),  g®q(a>))  <  e^(fq((i)),  gj|^q(oj));  V  g^qC***)  e  S^q,  follows  again  from 
section  2  and  also  have: 

IT 


a  .X  \  yrt_\“l 


eiCfqCu),  gj^q (<»>))  =  (2if)  tr  /  g^qtw)  fqt“)  S^qC***)  da»  «= 

IT  2  ^ 

-  2Tr(  /  (X®(aj))"^  dw)  V  f  (X®(a)))"^  tr  f  (m)  du 
'-ir  ^  D.  ^ 


2ir|/  (Xq(u)))"^  da>)" 


^-l  ‘'t 
k 


^-l 

k 


2  2 

n  m  (Dp 

‘"V  ”1  ’ 

nV(D^) 

2tt  c» 


“  2ir(/  (Xq(a>))"^  dw)”^  ^ 

Z~1 

(X®(a)))"^  du))‘^  -  e(fq(a)),  g®q(«*))) 


Finally,  the  result  follows  from  the  last  string  of  relationships. 


C: 


6.  Conclusions,  Discussion 

In  this  paper,  we  considered  the  prediction  and  interpolation  problems  for 
vector  processes  with  Ill-specified  statistical  structures.  We  modeled  the  uncer¬ 
tainty  in  the  statistical  description  of  the  processes,  by  assuming  that  their 
spectral  density  matrices  lie  within  certain  classes  of  such  matrices.  Then,  we 
formalized  the  problems  as  games,  whose  saddle  point  solutions  were  found  for  two 


•  -  •a  •  -k*  h  ‘J*  *  m  •j  ®  •‘•m  'ji  •••>  •j'  V*  ^J*  V  V  %•  V  '•*  V  * 


>v 


specific  classes  of  multivariate  spectral,  classes.  The  first  such  class  (F  ) 
represents  a  linear  contamination  of  a  nominal  spectral  density,  and  it  Includes  an 
energy  constraint.  The  second  class  (F^)  is  represented  by  fixed  energy  on  a 
finite  nu]id>er  of  prespecified  frequency  quantiles. 

Both  the  Fj^  and  the  F^  classes  were  assumed  to  consist  of  absolutely  continuous 
spectra  only.  If  these  classes  are  allowed  to  Include  singular  spectra  as  well, 
the  saddle  point  solutions  found,  cannot  be  guaranteed  to  belong  to  the  appropriate 
spectral  classes.  There  Is  an  exception  for  the  class  F  ,  where  we  can  allow  the 

Ju 

nominal  spectrum  (but  not  to  have  singularities  at  a  certain  set  of 

points.  Then,  each  member  of  F^  has  singularities  at  exactly  the  same  points  as 
F^(a)).  The  results  that  we  obtained  can  be  readily  extended  to  Include  this  case. 
However,  when  Is  allowed  to  have  singularities,  then  a  solution  does  not  in 

general  exist.  For  the  latter  case  and  for  scalar  stationary  process  an  approximate 
solution  is  given  by  Hosoya  (1978) . 

All  the  derived  solutions  for  the  prediction  and  interpolation  games  correspond 

% 

to  the  eigenvalues  with  the  "flattest"  possible  tails,  or  equivalently  to  measures 
with  the  most  evenly  spread  energy.  For  the  F^  class  we  obtained  Identical  spectral 
density  matrices  for  both  the  prediction  and  the  interpolation  solutions,  which  are 
diagonal,  with  a  single  eigenvalue  that  is  piece-wise  constant. 

The  solutions  that  we  obtained  for  the  F^  class  are  not  unique,  and  we  just 
selected  the  simplest  possible.  The  solutions  for  the  F,  class  are  also  nonunique, 

la 

in  general.  All  such  solutions  attain,  however,  the  same  saddle  value  of  the  game. 
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